Interactions between colour and motion in image segmentation

BACKGROUND . An important early stage in visual processing is image segmentation, in which similar regions are grouped together and segregated from dissimilar regions, so that distinct objects ultimately may be located and recognized. In the natural world, objects are simultaneously characterized by colour, motion, texture and other visual attributes. How does the human visual system combine these attributes to segment the image? Although colour and motion information are conveyed by distinct functional streams from retina to visual cortex, there is increasing evidence for early and substantial cross-talk between the streams. Here, we explore psychophysical evidence for interactions between colour and motion in image segmentation. RESULTS . Observers performed forced-choice segmentation tasks on random-dot stimuli. The dots in the vertical target figure were distinguished from the background dots by a different distribution of speeds or colours. To explore interactions between motion and colour segmentation, we added motion noise to the colour signal (or vice versa) by assigning all dots speeds (or colours) drawn from one of several noise distributions. Motion noise severely affects segmentation by colour. Motion noise defined by a broad distribution of speeds degrades colour segmentation, but a two-speed motion distribution (half moving up, half moving down) facilitates colour segmentation. Control experiments prove that the facilitatory effect is not caused by integrating colour information over different frames, nor can it be explained by probability summation over the two planes of moving dots. Colour noise also affects motion segmentation, but under a more restricted range of conditions, and not in a facilitatory way. CONCLUSIONS . Colour and motion information interact at early stages during image segmentation, before decisions based on either cue in isolation are made. The robust bipolar effects of motion information on segmentation by colour indicate that the establishment of motion-defined surfaces takes primacy, and that such surfaces constitute important primitives for further processing.